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Introduction 


At the initial orientation, students have more opportunities to accept a 
new strategy for HOCS-oriented learning in Marzanos' taxonomy. To cultivate 
students’ founding of HOCS, more challenges exist with students’ textual 
designs. As a dynamic learning goal in the 21st century, many educators 
have pointed that HOCS-oriented learning becomes a meaningful develop- 
ment to find out students’ constructions of cognitive knowledge and global 
competitiveness (Ghani et al., 2017). Innovative learning, critical thinking and 
problem-solving are accepted as three dominant constituents of the context 
with the guidance of students’ HOCS development in chemistry education 
(Danczak et al., 2017; Ghani et al., 2017). It is more and more significant to 
trace out students’ profound development of scientific knowledge and pro- 
mote their hierarchical levels of reasoning skills in Marzanos’ four cognitive 
taxonomy -- retrieval, comprehension, analysis, and knowledge utilization, 
within their total learning processes. Therefore, the presentation of Marzano’s 
four cognitive levels allowed more students to assess their thinking skills 
more accurately than Bloom's revised taxonomy (remembering, understand- 
ing, applying, analyzing, evaluating, and creating) in educational objectives 
(Anderson & Krathwohl, 2001) when they got stuck on problems with limited 
reactant mental cognition (Marzano & Kendall, 2007). 

All students know that chemistry is one of the four basic constituents 
in the STEM (science, technology, engineering, and mathematics) fields 
to pave their comprehensive foundation for hierarchical thinking abilities 
step by step. Accordingly, there are many advantages on students’ STEM 
developments in recognitive abilities, critical thinking, and problem-solving 
(Cordray et al. 2009; Toledo & Dubas, 2016). The ideal education of science 
goal should put emphasis on solving students’ conceptual problems and 
getting meaningful knowledge in terms of hierarchical levels of reasoning 
skills. Students need to accept their finding of scientific argumentations and 
construct hierarchical explanations during their chemistry learning process 
(Norris & Philips, 2012). It becomes more and more urgent tendencies for 
students to cultivate new HOCS-oriented learning in their scientific problem- 
solving abilities (Lopez et al., 2014). 
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Abstract. This research focuses on stu- 
dents’ higher-order cognitive skill (HOCS)- 
oriented learning to construct effective 
hierarchical thinking abilities in their 
chemical particulate nature of matter. For 
in-depth knowledge and profound under- 
standing, this research deals with students’ 
positive developments towards HOCS with 
a special guidance to Marzanos’ taxonomy. 
The methodology starts from the retrieval 
and comprehension of HOCS-centered as- 
sessment instrument to students’ analysis 
and knowledge utilization of transitional 
performances. 326 participants were as- 
signed to take natural science curriculum 
in 2019 academic year. With the help of 7 
scholars’ implements, the importance of 
content validity and inter-rate reliability 
were constructed through instrument 
developments of students’ HOCS perfor- 
mances. All research results indicated that 
more distributions of HOCS thinking abili- 
ties reached up to the target response of 
38.0% analysis task and 27.30% knowledge 
utilization in Marzanos’ spectrum. This 
research is beneficial and advantageous 
that students will be capable of additional 
learning efficiency in their multiple interac- 
tive engagements with academic resources 
and suggestions from instructors, research- 
ers, and science educators. 
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The current proposal for students to foster their HOCS-oriented chemistry text and evaluation instrument 
needs to set up a background environment for instructors’ profound deliverance of chemistry knowledge and 
conception understanding. In contrast to traditional old-fashioned instructions, the innovations of HOCS-oriented 
learning not only guide students’ problem-solving with algorithmic rules but also activate new conceptual knowl- 
edge of chemistry fields (Bransford et al., 2004; Domin & Bodner, 2012). Zoller and Pushkin (2007) have classified 
clear illustrations into dual HOCS combinations of situational knowledge and strategic knowledge. Students are 
required to make their accumulations of dual knowledge in order to construct HOCS-based hierarchical levels of 
reasoning skills in their chemistry learning. 

Most functionally of all, students should master their strategic developments of reasoning context and argument 
ability as an important presentation of HOCS-oriented chemistry learning in particulate nature of matter (PNM). The 
designs of these strategic developments should go steady with students’ abilities of critical thinking in their meaningful 
chemistry learning process (Danczak et al., 2017). Three important factors, innovative learning, critical thinking and 
problem-solving (Danczak et al., 2017; Ghani et al., 2017; Zoller & Pushkin, 2007) are included in the textual design for 
students’achievements of HOCS-oriented learning. For this purpose, students would be associated with HOCS-based 
hierarchical levels of reasoning skills as the essential knowledge of PNM basic conceptual understanding (Vachliotis 
et al., 2014). Discussions of stimulating students’ motivations and interests will be included as their constructions of 
HOCS-centered chemistry learning with Marzano’s taxonomy for hierarchical levels of chemistry equilibrium. 


Cognitive Levels of Marzano’s Taxonomy 


Many instructors, science educators, and researchers have pointed out the subtle aspects of students’ HOCS- 
oriented development in chemistry education (Vachliotis et al., 2014; Su, 2017, 2020). Essential learning objectives 
are needed for students to construct higher-order thinking abilities and problem-solving implements in chemistry 
equilibrium. There are four distributions of students’ hierarchical cognitive levels -- retrieval, comprehension, analysis, 
and knowledge utilization in Marzano’s spectrum (Marzano & Kendall, 2007) with brain-based learning. Toledo and 
Dubas (2016) unanimously noticed that Marzano’s taxonomy could describe students’ cognitive levels of lower-order 
and higher-order thinking abilities clearly with the framework of spectrum. Students’ lower-order thinking abilities 
included both retrieval and compression tasks during which students were required to obtain the rule-based learn- 
ing, the accurate reasoning information and integrated comprehensive understanding. Their higher-order thinking 
abilities included both analysis and knowledge utilization tasks during which students were required to fulfill the 
creative-based learning, the dual HOCS combination of situational knowledge and strategic knowledge. 

Students were required to analyze their new responses after extensive relations have been added to their 
knowledge appropriately within the third HOCS stage of Marzanos’ spectrum analysis task. Knowledge utilization 
belonged to the fourth HOCS stage during which students explained more authentic task, transferred new knowledge 
and raised new learning surrounding by the fulfillment of test items in combinations of situations and strategies 
(Zoller & Pushkin, 2007). Students’ chemistry learning did not come up to be a final stage of HOCS if they didn’t build 
their basic knowledge up to the successful stage of LOCS comprehension task. Students’ hierarchical cognitive levels 
were designed to promote their thinking ability by four distributions of Marzanos’ spectrum (Marzano & Kendall, 
2007). The fulfillment of this research was based on the framework of spectrum distributions to represent the HOCS- 
centered assessment instrument and guide students to achieve the learning goals of higher-order thinking abilities 
in chemistry equilibrium. 


Diagnostic Evaluation in Chemistry Equilibrium 


Margel et al. (2007) have found that many student's comprehensive evaluation toward PNM was difficult in 
easily generating more obstacles for learning misconception. A complete PNM understanding of students’ evalu- 
ations is critical to construct fundamental chemistry equilibrium knowledge and inspire confidence in students’ 
HOCS-oriented learning. As a result, there are functional assessments of research results for students’ misconceptions 
in chemistry equilibrium (Jaber & BouJaoude, 2012; Yezierski & Birk, 2006). Chandrasegaran et al. (2007) claimed 
that students’ misconceptions have limited their chemistry capability of description and interpretation in class. 
Such students’ limited capability would hinder their HOCS learning for more cognitive levels of comprehensive 
understanding. Therefore, Nyachwaya et al. (2011) have pointed that particle equilibrium was a very important 
comprehensive foundation of cognitive levels for students’ HOCS-oriented learning. 
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Students’ diagnostic evaluation and assessment had been designed and conducted in the chemistry class 
to provide teachers information on students’ comprehensive understanding in what they were required to do for 
HOCS-oriented learning (Green & Johnson, 2010). Cheung (2011) used diagnostic assessment in helping both pre- 
service and in-service teachers’ comprehensive understanding about the chemistry of lead-acid battery. Nitko and 
Brookhart (2011) found that students’ diagnostic evaluation could confirm their potential reasoning knowledge and 
proficient skills in chemistry. The assessment would be designed from students’ pretest or questionnaire to carry 
out their essential demand and capability and to obtain results from observation, analysis of data and learning. 
Using diagnostic assessment would be helpful for instructors in enhancing students’ conceptual understanding 
toward meaningful learning with individual guidance. To be a dominate factor in identifying students’ existing 
chemistry knowledge (Potgieter et al., 2010), prior knowledge provided a profound foundation for students’ sys- 
tematic investigation with chemistry conceptual development (Potgieter et al., 2010). It would be appropriate for 
researchers to design systematic test items combined with the diagnosis evaluation in order to assess students’ 
comprehensive HOCS understanding in chemistry equilibrium. 

In summary, this research had included a detailed assessment for HOCS-centered evaluation instrument to 
assess students’ comprehensive understanding. The employment of Marzano’s taxonomy paved a reliable platform 
for students to perform four distributions of hierarchical cognitive levels (retrieval, comprehension, analysis, and 
knowledge utilization) in spectrum. Students were ensured to motivate in building up their cognitive performances 
with comprehensive coalition and scientific concepts. 


Purpose and Research Questions 


Based on the above assumption, this research proposed a HOCS-oriented learning strategy for students to 
take active participations with their alternative assessment in chemistry equilibrium. To promote their thinking 
abilities, students were encouraged to do problem-solving, critical thinking and innovative learning in accordance 
with their understanding of hierarchical Marzano’s taxonomy assessment instrument. Three basic criteria were 
proposed as the research questions in the following way: 

(1) How to design the HOCS-oriented test items suitalble for students’ chemistry thinking abilities by Marzano’s 
taxonomy? (2) How to construct the validity and reliability of HOCS-oriented assessment instrument for students’ 
higher-order thinking abilities? (3) How to evaluate students’ Marzano cognitive understanding levels hierarchically 
in chemistry equilibrium for HOCS-oriented assessment instrument? 


Research Methodology 
General Background 


The basic framework of the HOCS-centered diagnostic evaluation instrument (HODEI) follows four hierarchical 
levels in Marzano’s taxonomy spectrum. Four hierarchical cognitive levels are included as retrieval, comprehension, 
analysis, and knowledge utilization shown in Figure 1 (Toledo & Dubas, 2016). The detailed discussions of four 
hierarchical cognitive levels, consist of level 1 (L1) as a retrieval task, level 2 (L2) as a comprehension task, level 3 
(L3) as an analysis task and level 4 (L4) as a knowledge utilization task. As the first basic level of the retrieval task, 
students were required to get needed information exactly through their recognizing, recalling, and executing as 
their feedback of learning objects. Since the work of the retrieval task didn’t treat working memory only to reproduce 
exactly the same algorithm, without acquiring any significant cognitive control at all, the retrieval task should be 
attributed to the lower-order thinking skill according to Marzano (Toledo & Dubas, 2016). 

The next discussion comes up L2 as the second level of students’ comprehensive task. For the required 
comprehensive work, students would explain the reasoning question, deduce symbolizing conclusion, and make 
appropriate sense of relating science knowledge. At the L2 of the comprehensive task, what students handled 
were simply working memory and existing knowledge as part of apprehensive meaning, which all pertained to 
the comprehensive level of the lower-order thinking skill. When students approached L3 as an analysis task, they 
would specify their expectant reasons for generalizing, analyzing, classifying, and finding out conceptual knowl- 
edge with alternative relations and applications. From the perspective of Marzano’s taxonomy, L3 of the analysis 
task pertained to the higher-order thinking skill. 
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Figure 1 


Cognitive levels by Marzano’s taxonomy 
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Another final investigation of the higher-order thinking skill, L4 as a knowledge utilization indicated that 
students’ investigative experiments, solving problems and a knowledge decision help in fostering their learning 
objectives (see Table 1). Therefore, students aimed at engaging in ultimate knowledge utilization from the existent 
knowledge to more authentic and creative science information. 


Table 1 
Students’ learning objectives with scores of Cognitive Level tasks in CE 


Task Score Learning Objectives 


Students can list characteristics for recognizing, recalling, and executing as their feedback of learn- 





Retrieval (L1) 1 ing objects. 
Comprehension (L2) D $ udents can clearly explain the reasoning question, deduce symbolizing conclusion, and make 
appropriate sense of relating science knowledge. 
: Students can specify their expectant reasons for generalizing, analyzing, classifying, and finding out 
Analysis (L3) 3 : : i aes 
conceptual knowledge with alternative relations and applications. 
Students can manage to investigate experiments, solve problems, and make a knowledge decision 


Rrigwlerige uulizauiore (4) : in fostering their learning objectives. 


Note: Learning Objectives were modified from Marzano and Kendall (2007). 


Participants of Research and Ethical Approval 


A total sampling of 326 participants (including sex distribution 118 males and 208 females; aged distribution 
from 18 to 21) were selected through two stages of qualification tests as total research sampling from Chung Yuan 
Christian university, Taiwan, in this research. There were 193 students who took part in the pilot study for develop- 
ments with the pre-knowledge of basic chemistry equilibrium at the first test stage. Other 133 students engaged 
in the assessments of the experimental research with HOCS-centered performances at the second stage as the 
required sample of research. All 326 participants were volunteers with suggestive findings and the anonymous 
results may be published. Ethical approval was obtained for this research (Taber, 2014) in 2019. 

Students’ discussion of Marzano’s taxonomy consisted of their thinking skills during learning processes precisely 
classified as both higher-order and lower-order cognitive levels, shown in Figure 1 and Table 1. At the lower-order 
cognitive levels, students would only acquire and understand the meaningful knowledge entirely, which included 
retrieval and comprehension tasks. Next for higher-order cognitive levels, students would explore both analysis 
task and knowledge utilization task with creative information. By Marzano’s taxonomy, students would construct 
a full-scale learning theory between higher-order and lower-order cognitive levels. Basing on Marzano’s taxonomy 
in chemistry learning, the author set up students’ design of the diagnosis evaluation instrument to discover 133 
students’ four hierarchical levels of thinking abilities in this research. 
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Instrument Design 


As an important instrument design, the draft test items of HODEI were adapted for as the assessment resources 
of chemistry textbooks (Brown, et al., 2018, pp. 666-707). All 25 test items of the first draft had already been scru- 
tinized, deleted and revised by six senior chemistry professors to provide the content validity of students’ HODE| 
test items. They examined and revised all test items in the aspects of the fluencies, correctness, and Marzano’s 
four hierarchal levels. The final results indicated that five test items were deleted with discriminated quality and 
analogical reasons. To be as functional as possible, all test items would provide correct results without making the 
mistake to avoid trying the probability that tenders the result linear programming (Griffard & Wandersee, 2001) 
during the whole editing processes. Furthermore, six renowned chemistry professors also stressed the crucial role 
of the corresponding HODEI test items with more exact analyses of inter-rater reliability as Kendall’s coefficient of 
concordance (w). The most influential source for pilot assessment results indicated that 193 university students 
gathered validity data and marked their major responsive quality in each HODEI test item. 


Research Procedures 


It was the primary concern to put students’ quantitative approach on major statistical findings with their minor 
development of the qualitative narration. The procedures of constructive designs are summarized in Figure 2 with 
evaluated HODEI instrument for students’ conceptual knowledge. Effective administrating pretest and post-test 
were assigned in accordance with the chemistry learning objectives within five weeks 2019 academic syllabus. 


Figure 2 
Flowchart of the Research 
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Data Analysis 
For acomplete analysis of HODEI implementation, this research conducted a series of statistical analysis such 
as, inter-rater reliability(w) and one-way ANOVA by software 22 SPSS for MS Windows to analyze each test item in 
this instrument. 
Research Results 


Quality of Test Items 


The major consideration for the validity and reliability of HODEI, the difficulty index of this study confirmed 
test items p value to examine 193 students’ pilot results as shown in Table 2. The main statistics of p values followed 
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the following 4 consistent results of four difficulty indices: (1) p < 30%, up to 9 difficulty learning test items, (2) 30% 
< p <50%, to 9 difficulty towards easy test items, (3) 50% < p < 70%, to 1 easy towards difficulty test item, and (4) 
70% <p, to 1 easy test item. The statistical p values marked four demonstrations of difficulty indices: each separate 
pvalue was measured difficulty or difficulty towards easy test items of 36.84% and towards as easy or difficulty test 
items of 13.16%, cumulative scores for test items in students’ total conceptual understanding level. All p values 
exemplified special characteristic functions on students’ understanding distributions of difficulty index in 20 test 
items of HODEI. Students’ difficulty indexes of pilot test results were compiled to find homogeneous distributions 
which would be helpful to detect their learning performances and individual differences. 


Table 2 
193 Students’ answering right rate for HODEI with cognitive levels 


Test item 1 2 3 4 5 6 7 8 9 10 1 #12 #13 #14 «15 «+16 «17:«©18~«©619 = «20 





COON i oR RO HB SR RR a SIRO SR RS a Ge a 
Level 
Ani 44 46 11 #34 «80 2 45 «48 32 «41 «+41 «#32 +17 «20 2 9 2 St 2 2 
Rate (%) 


Note: R, Retrieval; C, Comprehension; A, Analysis; K, Knowledge utilization. 


In students’ answering responses of HOCS options in 20 test items, six senior chemistry professors indepen- 
dently had evaluated the fulfillment for assigning the differentiated validity of HODEI Marzano’s cognitive levels. 
The Kendall's coefficient of concordance (w) was used to examine the responses of students’ statistic agreement 
by six senior chemistry professors. The research finding w value 0.669 (x?=76.317, p< .001) indicated that the sub- 
stantial responses of HODEI options in Marzano’s cognitive levels were on the agreeable affiliation by six senior 
chemistry professors. The exact analyses of inter-rater reliability opened up the statistic coefficient w between .6~.8 
which was developed to be consistent agreement (Marozzi, 2014). Therefore, the HODEI included a set of 20 test 
items which were published at Chinese Internet station assigned to estimate college students’ Marzano’'s cognitive 
understanding level (web address: https://goo.gl/forms/64aipoNQnMFGnP1F3). 


Students’ Answering Rate 


Figure 3 
Summary of HODEI Instrument in No. 16 Test Item 


16. In a closed vessel, put into equal amount of the gas particulate matter A and B respectively, just 
as the following figure, when the gas particulate matter A and B were mixed in producing the 
equilibrium equation as follows: Aj + Be = Ce + Di K=25. Which of the following particulate 
matter C and D in chemistry equilibrium will be the correct choice item? 


K=25 
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One more demonstrated example for No. 16 test item was shown in Figure 3 as one of correct choices in HODEI 
options. As shown from the summary of HODEI instrument for No. 16 test item in Figure 3, Marzano’s higher-order 
cognitive test item indicated the simple chemistry equilibrium descriptions of two gas particulate reactants of A 
and B to produce gas 

products of C and D. As the process went on, the equilibrium reaction A. + B= C, + Di, the equilibrium- 
constant expression was K= [C][D]/[B][A]. The aforementioned 20 test items provided a well-designed and profound 
understanding of learning for students to take active participation in HODEI options for guidance. Students could 
identify their own subsequent test items to develop from the Marzano’s hierarchical cognitive understanding. 

In students’ response to the research question 3, their answering right rates for assessment instrument were 
demonstrated in Table 3. All 133 students’ learning performance followed the proportions of 4 main Marzano 
distribution levels in hierarchical cognitive understanding. Students’ comprehensive extent of HODEI with cogni- 
tive levels was described in the following ways: students with the knowledge utilization level (L4) had 6 test items 
and their average answering rate 27.3%; students with the analysis level (L3) had 10 test items and their average 
answering rate 38.0%; students with the comprehension level (L2) had 4 test items and their average answering 
rate 25.8%, and students with the retrieval level (L1) got 0 test item in Table 3. 


Table 3 
133 Students’ Answering Right Rate with Cognitive Levels in Marzano’s Spectrum 


Test item 1 2 3 4 5 6 7 8 9 10 1 #12 13 #14 «15 #+%16 «#17 «18 «619 = 20 





Coan? ak ear pi aS aR a Re SR aS SRO AR ES EK OR RK 
Level 
Answer 
Rate) “2 38 17 34 Sf 12 46 41 95 42 St 32 15 22 18 15 21 50 29 2 


For further developments of their evaluation results, students should have complete command of two higher- 
order Marzano’s cognitive levels (L3 and L4). Students with the analysis level (L3) could concretely describe their 
expectations, explain reasons in generalizing, analyzing, classifying to find out any learning performances which 
could be matching to extend chemical equilibrium concept knowledge. For more intermediate students with the 
knowledge utilization level (L4), they could investigate, experiment, solve problems and make decisions in creat- 
ing anew understanding and cleared knowledge for more explanations to articulate authentic tasks of chemistry 
delivery and up to so called the Marzano’s higher-order cognitive level. Subsequently, both two higher-order 
cognitive levels (L3 and L4) contributed a differentiated rated proportion of 55.3% higher-order cognitive skills, in 
contrast with the above description of students’ low-order comprehensive level 25.8% (shown in Table 4). Accord- 
ingly, both students’ two higher-order Marzano’s cognitive levels (L3 and L4) were in accordance with the HODEI 
of cognitive levels as a HOCS-center hierarchical understanding. 


Students’ Particle Distributions of Cognitive Levels 


With macroscopic, symbol and micro characteristic presentations, students’ three particle distributions were 
summarized in Table 5. The principal classifications for students’ two different levels were assessed in their as- 
sembled characteristic particles of macroscopic, symbol and micro presentations in the Marzano spectrum. For 
students with higher-order thinking level, the array of their six test items were distributed in accordance with three 
distributions of particle characteristics. As suggested by results of HODEI, there were ten test items for students’ 
different distributed indications of the same thinking level in accordance with two presentations of symbol and 
micro characteristic particles. For students with lower-order thinking level, the array of their four test items were 
distributed in accordance with only two presentations of macroscopic and symbol characteristic particles. 
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Table 4 


133 Students’ Average Answering Right Rate (%) for HODEI with Cognitive Levels 


A Locs HOCS 
Cognitive 
Level 
R c A . 
Average answering 0 25.8 38 27.3 


right rate 


The total compilation of students’ three distributions of particle characteristics presented the following HODEI 
proportion: 50% test items with macroscopic particles, 100% test items with symbol particles, 80% test items with 
micro particles. From the above results, students equipped with a good command of macroscopic, symbol and 
micro particles would have a close link with the construction of HOCS-centered learning as the suggestions of 
Treagust et al. (2003) for comprehensive chemistry problem-solving (Su, 2017; Toledo & Dubs, 2016). 


Table 5 
The Test Items Distribution of the Macro, Micro, and Symbolic Levels in Chemistry 


Test item 1 2 3 4 =«5 6 7 8 9 10 11 #12 #13 #14 «15 «16«6«170«©18~«€©19 20 





COUN Re A A A a OLA ea ie se eR A 
Level 
Macro VoVoVoVoVeY vov V V 


Symbolic VV Vv Vv VO VO VO VO VO VO VO VO VO VO VO VO VO VO OV Y 


Micro VY V V 0 0 VY V VV VV 0 1 V VV VV 


Discussion 


Students’ validation of HODEI test items was in accordance with the finding of six senior chemistry profes- 
sors to present expert content validity and inter-rater reliability specifically. Most important of all, Marozzi (2014) 
proposed four agreements for Kendall’s coefficient of concordance of inter-rater reliability, in its functional relation- 
ship with the application of this methodology. In this research, the HODEI test items served as an effective instru- 
ment with consistent agreement for functional reliability and expert validity. It was scholarly agreement that the 
consistent agreement required students’ subsequent self-development of HOCS-centered learning for the HODEI 
understanding with Marzano spectrum (Toledo & Dubas, 2016). Furthermore, research studies with the cognitive 
hierarchical HODEI test items would clarify students’ four understanding tasks starting from lower-order cogni- 
tive levels (retrieval task and comprehension task) to higher-order cognitive levels (analysis task and knowledge 
utilization task) in Marzano spectrum. 

Students’engagement of answering right rates was generally assigned as an active response of HODEI test items 
in Table 4. They could achieve average answering right rate both for 55.3% students with higher-order cognitive 
skill and 25.8% students with lower-order cognitive skill in this research. It provided a new evaluation and assess- 
ment for college students to focus on their research-based development of Marzano’s spectrum within chemistry 
learning. Within contemporary reforms of chemistry teaching, scholars emphasized an existing consensus for 
students’ acquisition of HOCS to constitute a dynamic instructional goal (Toledo & Dubas, 2016; Zoller & Pushkin, 
2007). Ghani et al. (2017) pointed that the HOCS development of educational transformation in the 21st century 
has promoted students’ in-depth knowledge understanding in science learning. Seen as an important functional 
indication, students’ HOCS-oriented achievement should go hand in hand with HOCS chemistry teaching goals to 
promote more students’ constructive and transferable knowledge. 
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Reaching their engagement upon three distributions of particle characteristics, students were available for 
their presentations of 50% test items with macroscopic particles, 100% test items with symbol particles, 80% test 
items with micro particles. This research offered an effective functional way to distinguish particle distributions of 
lower and higher-order cognitive levels in Marzano’s taxonomy which could promote students develop these skill 
tasks gradually as proposed by Toledo and Dubas (2016). In addition to the finding by Jaber and BouJaoude (2012) 
for macro—micro-symbolic teaching in promoting students’ latent conceptual understanding and relational learn- 
ing of chemical reactions, the claims for thermodynamic properties by Becker et al. (2013) served as an alternative 
justification of students’ particulate-level of understanding physical and chemical properties. The improvement of 
Su’s (2020) proposal for micro and symbolic performances also animated students’ cognitive skills and advanced 
their hierarchical understanding in chemistry equilibrium classroom. 

The application of Marzano’s Taxonomy with HOCS-oriented learning testified a special perspective to assess 
students’ higher-order thinking abilities in chemistry learning. The required achievements of Marzano's four cogni- 
tive levels were different from those of Bloom’s revised taxonomy in educational objectives (Anderson & Krathwohl, 
2001; Marzano & Kendall, 2007). As Bransford et al. (2004) suggested that HOCS-oriented learning was essential 
in transferring more easily students’ authentic knowledge across chemistry courses and to apply science to new 
situations, the combination of instructive situations and strategies by Zoller and Pushkin (2007) made an advanced 
appeal for students’ implementation of higher-order cognitive skills step by step. 

As an important foundation of students’ STEM, chemistry was inseparable from new problems which schol- 
ars encountered in clinching aggressively with Marzano’s HOCS-oriented learning (Eichler& Peeples, 2016; Norris & 
Philips, 2012). The major hierarchical spectrum of Marzano’s taxonomy was constructed in four classifications from 
retrieval, comprehension, analysis to knowledge utilization as a functional method to distinguish lower-order from 
higher-order thinking abilities (Toledo & Dubas, 2016). The innovated analyses of Marzano’s taxonomy were much 
different from those of simple three-tier diagnostic instrument and order multiple choice items in traditional teach- 
ing assessments (Su, 2019). The most successful target of Marzano's taxonomy would be associated with students’ 
experience and participation in a new coalition of higher-order thinking abilities systematically. 


Conclusions 


The final results of this research proposed a well-prepared development for students’ participations in Marzano's 
hierarchical spectrum and their unification in higher-order thinking competence. Through the step by step composed 
learning of Marzano’s taxonomy, students succeeded in constructing higher-order thinking abilities related to HOCS- 
oriented and Marzano's brain-based learning for the particle nature of matter in chemistry equilibrium. In regard 
to students’ particle distribution, it also became important to take into account their answering rates, as well as the 
academic alignment of positive affect. Students’ engagement of hierarchical understanding originated from their 
comprehensive participation in chemistry classes andtothefra | meworkoflong-term STEM in Marzano’s spectrum. 

This research would inspire students to make more frequent cognitive participation activities shifting from 
comprehensive analysis to knowledge utilization in contrast with most traditional chemistry learning. Students 
would strengthen their hierarchical development and multiple interactive engagements with HOCS-oriented ac- 
tivities of Marzano’s brain-based learning for promoting more constructive thinking elements of HOCS chemistry 
abilities. In accordance with scholars’ requirements for thinking processes of high-achieving programs, this research 
offered a new HOCS-centered perspective for students’ cognitive participation and the future design of academic 
resources and suggestions in chemistry classroom. 

On the top finding of that statement, despite not aimed at discussing chemical particulate nature of matter 
regarding students’ learning progress individually, the Marzano’s HOCS-oriented learning was expected to be 
beneficial and advantageous for students’ additional learning efficiency. More suggestive instructions for students’ 
HOCS-centered perspective of both analysis and knowledge utilization tasks in Marzano’s spectrum would be 
responsive consideration to instructors’ teaching, professional development, and future research. 


Acknowledgements 
The author's sincere thanks must also be given to the patronage of the Ministry of Science and Technology, 


MOST in Taiwan (under Grant No. MOST 107-251 1-H-237-001). Without their help and financial support, this research 
could not have been completed in the present form. 


313 


https://doi.org/10.33225/jbse/21.20.305 LT 


Journal of Baltic Science Education, Vol. 20, No. 2, 2021 


A NEW ASSESSMENT OF HOCS-ORIENTED LEARNING FOR STUDENTS’ HIGHER-ORDER ISSN 1648-3898 /Print/ 
THINKING ABILITIES BY MARZANODO’S TAXONOMY 

(PP. 305-315) ISSN 2538-7 138 /oniines 
References 


Anderson, L. W., & Krathwohl, D. R. A. (2001). Taxonomy for Learning, Teaching, And Assessing: A Revision of Bloom’s Taxonomy of 
Educational Objectives. Longman. 

Becker, N., Rasmussen, C., Sweeney, G., Wawro, M., Townse, M., & Cole, R. (2013). Reasoning using particulate nature of matter: 
An example of a sociochemical norm in a university-level physical chemistry class. Chemistry Education Research Practice, 
14, 81-94. https://doi.org/10.1039/C2RP20085F 

Bransford, J. D., Donovan, M. S., & Pellegrino, J.W. (2004). How People Learn: Brain, Mind, Experience, and School. In Donovan, M. 
S., Bransford, J. D., & Pellegrino, J. W., Commission on Behavioral and Social Sciences and Education. National Academy Press. 

Brown, T. E., Lemay, H. E., Bursten, B. E., Murphy, C., Woodward, P., & Stoltzfus, M. E. (2018). Chemistry: The Central Science (14th 
Edition). Malaysia Press. 

Chandrasegaran, A. L., Treagust, D. F., & Mocerino, M. (2007). The development of a two-tier multiple-choice diagnostic instrument 
for evaluating secondary school students’ ability to describe and explain chemical reactions using multiple levels of 
representation. Chemistry Education Research and Practice, 8(3), 293-307. https://doi.org/10.1039/B7RP90006F 

Cheung, D. (2011). Using diagnostic assessment to help teachers understand the chemistry of the lead-acid battery. Chemistry 
Education Research and Practice, 12(2), 228-237. https://doi.org/10.1039/C1RP90028E 

Cordray, D. S., Harris, T. R., & Klein, S. (2009). A research synthesis of the effectiveness, replicability, and generality of the 
VaNTH challenge- based instructional modules in bioengineering. Journal of Engineering Education, 98(4), 335-348. 
https://doi.org/10.1002/j.2168-9830.2009.tb01031.x 

Danczak, S. M., Thompson, C. D., & Overton, T. L. (2017). What does the term Critical Thinking mean to you? A qualitative analysis 
of chemistry undergraduate, teaching staff and employers’ views of critical thinking. Chemistry Education Research and 
Practice, 18(3), 420-434. https://doi.org/10.1039/C6RP00249H 

Domin, D., & Bodner, G. (2012). Using students’ representations constructed during problem solving to infer conceptual 
understanding. Journal of Chemical Education, 89(7), 837-843. https://doi.org/10.1021/ed1006037 

Eichler, J. F., & Peeples, J. (2016). Flipped classroom modules for large enrollment general chemistry courses: a low barrier 
approach to increase active learning and improve student grades. Chemistry Education Research and Practice, 17 (1), 197- 
208. https://doi.org/10.1039/C5RP00159E 

Ghani, |. B. A., Ibrahim, N. H., Yahaya, N. A., & Surif, J. (2017). Enhancing students’ HOTS in laboratory educational activity 
by using concept map as an alternative assessment tool. Chemistry Education Research Practice, 18(4), 849-874. 
https://doi.org/10.1039/C7RP00120G 

Green, S. K., & Johnson R. L. (2010). Assessment is essential. New York. 

Griffard, P. B., & Wandersee, J. H. (2001). The two-tier instrument on photosynthesis: What does it diagnose? International Journal 
of Science Education, 23(10), 1039-1052. https://doi.org/10.1080/095006901 10038549 

Jaber, L.Z., & BouJaoude, S. (2012). A macro—micro-symbolic teaching to promote relational understanding of chemical reactions. 
International Journal of Science Education, 34(7), 973-998. https://doi.org/10.1080/09500693.201 1.569959 

Lopez, E., Shavelson, R. J., Nandagopal, K., Szu, E., & Penn, J. (2014). Factors Contributing to Problem-Solving Performance in 
First-Semester Organic Chemistry. Journal of Chemical Education, 91(7), 976-981. http://doi.org/10.1021/ed400696c 

Margel, H., Eylon, B. S., & Scherz, Z. (2007). A longitudinal study of junior high school students’ conceptions of the structure of 
materials. Journal of Research in Science Teaching, 45(1), 132-152. https://doi.org/10.1002/tea.20214 

Marozzi, M. (2014). Testing for concordance between several criteria. Journal of Statistical Computation and Simulation, 84 (9), 
1843-1850. https://doi.org/10.1080/00949655.2013.766189 

Marzano, R. J., & Kendall, J. S. (2007). The New Taxonomy of Educational Objectives. (2nd ed). Thousand Oaks, CA. 

Nitko, A. J., & Brookhart, S. M. (2011). Educational assessment of students (6th ed.). Boston, MA. 

Norris, S. P. & Phillips, L. M. (2012). Reading science: How naive view of reading hinders so much else. In A. Zohar & Y. J. 
Dori (Eds.), Metacognition in science education: Trends in current research (pp. 37-56). Dordrecht, The Netherlands. 
https://doi.org/10.1007/978-94-007-2132-6 

Nyachwaya, J. M., Mohamed, A. R., Roehrig, G. H., Wood, N. B., Kern, A. L., & Schneider, J. L. (2011). The development of an open- 
ended drawing tool: an alternative diagnostic tool for assessing students’ understanding of the particulate nature of matter. 
Chemistry Education Research and Practice, 12(2), 121-132. https://doi.org/10.1039/c1rp9001 7j 

Potgieter, M., Ackermann, M., & Fletcher, L. (2010). Inaccuracy of self-evaluation as additional variable for prediction of students at 
risk of failing first-year chemistry. Chemistry Education Research and Practice, 11(1), 17-24. https://doi.org/10.1039/C001042C 

Su, K. D. (2017). Tactic fulfillments of three correlations for problem-solving maps and animated presentations 
to assess students’ stoichiometry performances. Journal of Baltic Science Education, 16(5), 733-745. 
http://oaji.net/articles/2017/987-1509214073.pdf 

Su, K. D. (2019). A feasible guidance for ordered multiple-choice items in students’ hierarchical understanding levels. Journal of 
Baltic Science Education, 18(1), 77-89. https://doi.org/10.33225/jbse/19.18.77 

Su, K. D. (2020). Enhancing students’ high-order cognitive skills for hierarchical designs in micro and symbolic particulate nature 
of matter. Journal of Baltic Science Education, 19(5), 842-854. https://doi.org/10.33225/jbse/20.19.842 

Taber, K. S. (2014). Ethical considerations of chemistry education research involving ‘human subjects. Chemistry Education 
Research and Practice, 15, 109-113. https://doi.org/10.1039/C4RP90003K 

Toledo, S., & Dubas, J. M. (2016). Encouraging Higher-Order Thinking in General Chemistry by Scaffolding Student Learning 
Using Marzano’s Taxonomy. Journal of Chemical Education, 93(1), 64-69. https://doi.org/10.1021/acs.jchemed.5b00184 


314 


URS https://doi.org/10.33225/jbse/21.20.305 


Journal of Baltic Science Education, Vol. 20, No. 2, 2021 


ISSN 1648-3898 /Print/ A NEW ASSESSMENT OF HOCS-ORIENTED LEARNING FOR STUDENTS’ HIGHER-ORDER 
THINKING ABILITIES BY MARZANODO’S TAXONOMY 
ISSN 2538-7138 /oniines (Pp. 305-315) 


Treagust, D. F., Chittleborough, G. D., & Mamiala, L. T. (2003). The role of submicroscopic and symbolic representations in chemical 
explanations. International Journal of Science Education, 25(11), 1353-1368. https://doi.org/10.1080/0950069032000070306 

Vachliotis, T., Salta, K., & Tzougraki, C. (2014). Meaningful understanding and systems thinking in organic chemistry: 
Validating measurement and exploring relationships. Research of Science Education, 44(2), 239-266. 
https://doi.org/10.1007/s11165-013-9382-x 

Zoller, U., & Pushkin, D. (2007). Matching higher-order cognitive skills (HOCS) promotion goals with problem-based 
laboratory practice in a freshman organic chemistry course. Chemistry Education Research and Practice, 8(2), 153-171. 
https://doi.org/10.1039/B6RP90028C 

Yezierski, E. J., & Birk, J. P. (2006). Misconceptions about the particulate nature of matter — Using animations to close the gender 
gap. Journal of Chemical Education, 83(6), 954-960. https://doi.org/10.1021/ed083p954 


Received: November 20, 2020 Accepted: March 12, 2021 


Cite as: Su, K.-D. (2021). A new assessment of HOCS-oriented learning for students’ higher-order thinking abilities by Marzano’s 
taxonomy. Journal of Baltic Science Education, 20(2), 305-315. https://doi.org/10.33225/jbse/21.20.305 


King-Dow Su PhD, Professor, Department of Hospitality Management and Center for 
General Education, Hungkuo Delin University of Technology; NO.1, Lane 
380, Ching-Yun Road, Tu-Cheng District., New Taipei City, Taiwan 23646, 
R.O.C. & Center for General Education, Chung Yuan Christian University, 200 
Chung Pei Road, Chung Li District, Taoyuan City, Taiwan 32023, R.O.C. 
E-mail: su-87168@mail.hdut.edu.tw 
ORCID: https://orcid.org/0000-0001-5248-5589 


315 


https://doi.org/10.33225/jbse/21.20.305 LQ 


